Physicians have an important need for knowledge and information of the structure and condition of a patient's internal anatomy. More importantly, physicians have a need for such information which is obtained by non-invasive techniques. These needs were first addressed by the use of x-rays. In recent years, however, the x-ray machine has been supplemented in many situations by medical imaging techniques which yield three dimensional (3D) information. These techniques include, for example, computed tomography, magnetic resonance imaging, positron emission tomography, tomographic gamma scintillation imaging, and ultrasonic scanning.
Perhaps the best known technique of this type is computed tomography (CT), also known as computed axial tomography (CAT). With computed tomography, a complete three dimensional examination is made up of a sequence of two dimentional (2D) cross sections or "slices". Slice information is acquired by rotating a thin, fan shaped beam of x-rays about the long axis of the patient. Each slice is irradiated by its edges; the transmitted x-ray beams are captured with position sensitive radiation detectors and, thus, x-ray attenuation measurements are obtained from many different directions across the slice. The two dimensional images are reconstructed from these measurements by a mathematical procedure known as convolution and back projection. The output of the reconstruction procedure is an array of numbers (known as pixels in 2D, or voxels in 3D) representing the radiological density (x-ray attenuation) at points within the slice.
A recently developed imaging modality which does not employ x-rays is magnetic resonance imaging (MRI). This technique uses large magnetic fields to excite protons within the body through the phenomenon of nuclear magnetic resonance (NMR). The excited protons produce a radio frequency (RF) signal which can be position encoded. Three dimensional information can be built up slice by slice, as with x-ray CT. The two dimensional slices are reconstructed for viewing using mathematical procedures analogous to those used in x-ray CT. In MRI, the information associated with each voxel is a composite of proton density (essentially, the amount of water present) and so called T1 and T2 relaxation times, which are functions of physical and chemical structure.
Other three dimensional imaging techniques fall within the realm of nuclear medicine. The basic principle here is to introduce radioactive substances (radio pharmaceuticals) into the body, relying on their pharmacological properties for uptake into specific organs (for example, radioactive iodine can be used to label the thyroid). These radioactive substances produce radiation which may be measured with position sensitive detectors external to the body, known as gamma cameras. Two dimensional projected images (comparable to those obtained with conventional x-rays) can be generated with analog electronic circuitry. To obtain reliable three dimensional information, however, single photon emission computed tomography (SPECT) or positron emission tomography (PET) is employed, both of which rely on digital techniques. SPECT systems make use of routine gamma radiation emitting radiopharmaceuticals combined with tomographic scanning techniques and may be thought of as a tomographic gamma scintillation imaging technique. PET systems, in contrast, employ annihilation coincidence detection to detect positron annihilation radiation from positron emitting radiopharmaceuticals. In both of these modalities, the information associated with each voxel is a measure of the concentration of radiopharmaceutical at the corresponding volume increment within the patient. SPECT and PET differ from CT and MRI in that the images are fundamentally physiological rather than anatomical (although certain MRI studies have a physiological component). Thus, for example, nuclear medicine studies are used to demonstrate abnormal growth activity in bones which otherwise appear normal.
Another common imaging modality which yields three dimensional information in digital format is diagnostic ultrasound. This technique relies on the reflection of sound waves at interfaces within the body (e.g., between fluid and soft tissue) to generate echoes; the elapsed time between the transmission of a pulsed sound wave and the reception of its echo give a measure of the distance to the interface. Transmission ultrasound systems have been proposed, but these are currently in the experimental stage. Conventional ultrasound images are built up slice by slice in a manner analogous to CT (except that the slices are usually longitudinal rather than transverse); digital techniques are not needed to produce the images although almost all modern devices store the image in digital format to avoid electronic drift and to facilitate post processing. In this modality, the information associated with each voxel represents the strength of the echo at the corresponding point within the body; this in turn is a measure of acoustic impedance, a function of the type of materials present at the interface.
A major drawback of each of the imaging devices described above is that images are produced which comprise two dimensional slices of the internal anatomical structures being observed. Physicians must then mentally "stack" an entire series of these two dimensional slices in order to infer the structure of the three dimensional objects under investigation. Many problems are inherent in such an approach.
First, the interpretation of a series of stacked, two dimensional images by a physician requires a great deal of specialized knowledge and skill. Secondly, such an approach is extremely time consuming. Thirdly, the approach is prone to inaccuracy.
What is clearly needed is a medical display device which produces a three dimensional representation of internal anatomical structures produced from a full series of stacked two dimensional slices of that structure. Even more desirable is a medical image display device which provides the physician or other observer with the ability to manipulate the object and its image interactively in real time such that the object may be viewed from various directions and in various modes in real time. By real time display is meant that the video display output should be updated at or near video rates of 30 frames per second. Provided there is minimal or no delay between operator action and the corresponding change in the final image, this update rate would provide instantaneous perceptual feedback. It should be clear that such an interactive three dimensional display system permitting a physician to visualize and interact with a shaded three dimensional representation of an anatomical structure would greatly facilitate the examination of the structure in conjunction with medical research, clinical diagnoses, and the planning of surgical procedures.
A number of three dimensional display systems for medical objects have been described in the literature, but none of these provide realistic shaded images at the full resolution of the input data with real time interactive capabilities.
Three dimensional medical data sets can be displayed in the following ways: the data can be organized into a sequence of reprojected views or slices; it has been proposed to create true three dimensional images in space; and so-called two and a half dimensional (2.5D) images can be generated by projecting objects or object onto a two dimensional screen with depth cues given by shading.
Many computed tomography and magnetic resonance imaging display systems provide facilities to work through a sequence of two dimensional slices fairly rapidly, so that a trained physician can create a mental impression of the three dimensional structure. On the other hand, only the original slices captured by the imaging apparatus can be rapidly displayed. Reslicing or reformatting the image data to generate new two dimensional slices without re-imaging the patient, referred to as multi-planar reconstruction or MPR, slows the display process considerably.
True three dimensional images can be created in space using several different approaches. In one approach, a varifocal mirror is used to view slice images which are sequentially displayed on a cathode ray tube (CRT) under computer control. The mirror surface is vibrated in synchronism with the update of the CRT. Different images are seen as the mirror vibrates back and forth, giving a sense of depth. Another proposed approach employs a volume of a fluorescent gas, such as iodine-chlorine (I-Cl) vapor, which is excited to fluorscence by intersecting laser beams. The laser beams are scanned in a similar manner to the raster scanning of video monitors and television screens, except in a three dimensional sense.
The most familiar method of generating realistic images from a three dimensional scene is to project it onto a two dimensional screen and rely on motion parallax, projective geometry, shading, and hidden surface removal to create the illusion of depth. The result is similar to conventional television and motion pictures, which viewers readily intrepret as representing three dimensional scenes.